Joy-stick like graphical user interface to adjust 3D cross sectional plane in 3D volume

ABSTRACT

Methods for a joy-stick graphical control in 3D space are disclosed. 3D data are rendered with a certain orientation in a 3D rendering space within a region of a 2D screen. One or more cross sectional views of the 3D data are rendered in the 3D rendering space where the cross sectional views are derived based on corresponding one or more 3D planes cutting through the 3D data at a 3D location. On one 3D plane, a joy-stick control is rendered in the 3D space at a pose determined based on the joy stick.

CROSS REFERENCE TO RELATED APPLICATION

This application claims priority under 35 U.S.C. §119 from ProvisionalPatent Application No. 60/800,418, filed on May 16, 2006. The entiresubject matter of the application is incorporated herein by reference.

BACKGROUND

1. Technical Field

The present teaching relates to methods for data processing. Morespecifically, the present teaching relates to methods for 3D dataprocessing, visualization, and manipulation.

2. Description of Related Art

To explore dense 3D data set such as 3D volumes from medical scanningdevices like CT or MR, cross sectional slices are usually used. In mostof the nowadays applications, slices aligned with coordinate axes aregenerated as three orthogonal views to facilitate exploring the 3Dvolume. However, sometimes the slices along these angles may not revealfeatures users expect to see. in these situations, oblique ordouble-oblique angle slices are then needed. However, how to allow auser to effectively determine the location of the oblique ordouble-oblique slice is a non-trivial task.

To determine the location of an oblique or double-oblique slice is todetermine the orientation and position of a plane in a 3D space. Thereare six degree of freedom (DOF) associated with an orientation andposition of a plane. Some of the 3D input devices can provideinformation specifying six degrees of freedom instantly. However, in thecurrent computer environment, the keyboard, mouse and two-dimensional(2D) screen may still be the most ubiquitous input and output devices.The mouse used in those environments is a two DOF device. How can oneutilize such a device to accomplish a six DOF action is a challenge.Some applications implement each degree of freedom as a slider control.User may adjust each slider to change the value of each degree offreedom. However, this approach is very non-intuitive, time consuming,and difficult to use. The other approach is to provide three orthogonalaxis-aligned views and the intersection lines of the 3D plane with thesethree orthogonal planes. Users may drag or rotate these intersectionlines to define the new orientation and position of the 3D plane. Sincea user can see the underlying image in each view, it is also easier foruser to move the plane to the desired location. However, this approachstill requires that a user imagine the spatial relationship of the planewith the orthogonal planes which is not straightforward and it is hardto define a double-oblique plane. Due to drawbacks in the aforementionedapproaches, a more direct and intuitive solution is needed.

SUMMARY

The present teaching discloses a joy-stick like graphical user interfacecontrol in a perceptually 3D scene that allows a user to change theorientation and position of a 3D plane directly in the perceptually 3Dscene. This control stick comprises a plurality of control parts,including a stick tip, a stick body, and a stick base. The stick tip maybe used to change the orientation of the 3D plane, the stick body isused to shift the location of the 3D plane along a certain direction ofthe 3D plane, and the stick base is used to change a point on the 3Dplane that defines a base of rotation of the 3D plane.

A control part may be active or inactive. An active control part may beassociated with a visual indication such as being highlighted when mouseis moved over the control part to facilitate a user to choose to operatea control part that is currently active. Manipulation of a control partyields a control action that causes changes in data processing. Anoperation performed using a control part A control action may bedesigned to follow the movement of the mouse so that a user has fullcontrol of the action. With this intuitive control and supported views,user may adjust the 3D plane in a more effective way.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an exemplary display screen arrangement with a joy-stickcontrol, according to an embodiment of the present teaching;

FIG. 2 illustrates exemplary control operation using a joy-stickcontrol, according to an embodiment of the present teaching;

FIG. 3 illustrates an exemplary rotation range of a joy-stick control,according to an embodiment of the present teaching; and

FIG. 4 shows an exemplary use of a joy-stick control that iscontrollable on both sides of a 3D plane according to an embodiment ofthe present teaching.

DETAILED DESCRIPTION

The present teaching describes a What-You-See-Is-What-You-Got (WYSIWYG)approach for 3D data manipulation and processing control. With thisapproach, a user can see 3D data and a joy-stick control and operationthereof in a 3D rendering space directly. This approach eliminates theneed for a user to mentally reconstruct a 3D picture. In accordance withthe present teaching, a 2D display screen is used to render andmanipulate 3D data. On the 2D display screen, at least a portion of the2D display screen is used for 3D rendering. This portion may be called a3D rendering space. In the 3D rendering space, 3D data may be renderedin a certain orientation in accordance with a 3D coordinate system. Inaddition to the 3D rendering of the 3D data as a volume, there are oneor more 3D planes also rendered in the 3D rendering space. Each of the3D planes cuts through the 3D data volume in a direction along thesurface of the plane. These cutting planes may intersect with oneanother.

FIG. 1 illustrates an exemplary display arrangement 100 of differentregions on a 2D display screen according to an embodiment of the presentteaching. FIG. 1 shows a display space 106 and a plurality of viewingareas 107 a, 108 a, and 109 a in which a 2D images may be displayed. Inthe display space 106, 3D data 101 is rendered in a 3D rendering spacewhere a 3D plane 102 is visualized with the rendering of the 3D data andis oriented or cut through the 3D data with a certain angle in the 3Dvolume. In some embodiments, there nay be additional 3D planes, similarto plane 102, each of which has its own 3D pose and may slice throughthe 3D data in accordance with an angle in the 3D space. When there aremore than one cutting planes, they may have a certain spatialrelationship. For example, such cutting planes may form straight angleswith each other.

In the exemplary display arrangement 100, there may also be one or moreviewing areas. For example, there may be three viewing areas 107 a, 108a, 109 a, which may be designed to display cross sectional viewsobtained along a certain dimension or axis of the 3D data rendered inthe 3D rendering space 106. For example, the viewing area 107 may beused to display 2D cross sectional views or slices of the 3D data alongX axis. The viewing area 108 may be used to display 2D cross sectionalviews or slices of the 3D data along Y axis. The viewing area 109 may beused to display 2D cross sectional views or slices of the 3D data alongZ axis.

When a volume (or volumes) is rendered in a space in which a cuttingplane slices through, the corresponding cross sectional slice may alsobe displayed or projected on the cutting plane in the 3D rendering spacedirectly. In addition, the cross sectional view may also be displayed asa 2D image in the space where the 3D rendering is displayed. Forexample, in the display space 106, the 2D cross sectional view 102 acorresponding to the 3D plane 102 is displayed as a 2D image in the 3Drendering space.

When there are more than one 3D planes, such planes may have a certainspatial relationship. For instance, three planes may be orthogonal toeach other. They may or may not align with the three (X, Y, and Z) axesof the 3D space. The entire 3D rendering space 101 may be rotated via amouse-dragging operation. Multiple planes slicing through the 3D datamay intersect. Such 3D planes may be grouped together so that when onemoves, others also move accordingly. Such 3D planes may also beindependent so that changes in one plane may not have an effect onothers.

In FIG. 1, a joy-stick is rendered as attaching to a 3D cutting plane,e.g., the 3D plane 102. The joy-stick has multiple part, including astick base 103, a stick body 104, and a stick tip 105. The stick base103 is the part attached to a point on the 3D plane 102. The joy-stickis rendered within the confinement of the space in which the 3D datavolume is rendered. The joy-stick may be rendered in a fashion so thatits 3D pose has a certain spatial relationship with respect to the 3Dplane 102. For example, the joy stick may be rendered so that the stickbody 104 forms a certain angle with respect to the surface normal of the3D plane 102. One example is to be in alignment with the surface normalof the plane 102. As illustrated, the stick base 103 is where the joystick is attached to the 3D plane. In addition, the stick base 103serves as a point with respect to which the joy stick can be rotated viaa movement of the stick tip.

A user may manipulate the 3D plane 102 via the joy stick. This allowsthe user to visualize different cross sectional views of the 3D data ina flexible manner. For example, a user may change the orientation of the3D plane 102 by grabbing the stick tip 105 of the joy-stick to rotatearound the base point of the stick. In accordance with the presentteaching, when the joy stick is rotated in this manner, the orientationof the plane 102 changes accordingly. In addition, the user may slide upand down along the stick body. When the slide position along the stickbody changes, the 3D position of the plane 102 may change accordingly.Furthermore, the user may grab the stick base 103 of the joy-stick andmove around the surface of the plane 102. This may cause a change in theposition at which the joy stick is attached to the plane.

In accordance with the present teaching, the joy stick may be renderedas an object which has rigid parts that are connected in a certainphysical configuration. When one part of the joy stick changes, otherparts may change based on the relationship among different parts. Forexample, if the stick base is moved, the other parts of the joy stickmove accordingly. However, depending on the configuration of differentparts, in certain situations, movement of one part may not cause otherparts to move. For example, when the stick tip is moved or rotated, thestick base does not move.

When the pose of the plane 102 is changed, including orientation or 3Dlocation, the rendering of the plane 102 is updated accordingly. Forinstance, the cross sectional view of the plane 102 may be dynamicallyupdated. In addition, the 2D image display in region 102 a of the crosssection sliced by the plane 102 may also be updated on the fly.Furthermore, when there are more than one 3D cutting planes in the 3Drendering space, these planes may form a certain spatial relationship.In some embodiments, such spatial relationship among different planesmay be dynamically maintained, when the pose of one of the planeschanges. In this case, the rendering of other planes may also be updateddynamically. For example, if there are three planes orthogonal to eachother, if one changes the orientation, the orientations of other planesalso change in order to maintain the orthogonality. In this case, thejoy stick attached to one plane may impact other planes so that thecontrol over one plane via a joy stick can be extended to other planes.In some embodiments, even though there are multiple cutting planes inthe 3D space and they initially have a certain spatial relationship, thechanges made to one of such planes via a joy stick attaching to thatplane may not have impact on others. In some embodiments, each suchcutting plane may have its own control joy stick and the scope ofcontrol performed through each such joy stick may be applied to theplane to which the joy stick is attached.

FIG. 2 illustrate an exemplary operation of the joy stick, according toan embodiment of the present teaching. There are three parts of the joystick that can be manipulated to perform certain control operations. Forexample, a user may grab the stick tip and rotate around the stick basepoint, as shown in 201. As described herein, when the stick tip rotates,the orientation of the plane to which the joy stick is attached (e.g.,102) also changes. A grab operation may be performed via a mouseoperation such as holding down or one or more clicks. A user may alsouse a mouse to slide along the stick body, as shown in 202. A slidingoperation, either up or down, may cause accordingly the movement of theposition of the plane 102, as illustrated in 202. A user may also grabthe stick base via a mouse operation and move the stick base along theplane 102 to change the position of the joy stick on the plane. This isillustrated in 203. The joy stick may be implemented as a rigid objectso that when the base of the stick moves, the rest of the joy stick suchas stick body and stick tip also move accordingly.

In operation, the mouse is operable in a 2D space (the 2D screen) butits effect is 3D control over a 3D plane in a 3D rendering space. Thestick tip, the stick body, and the stick base may be activatedindividually and/or separately. For example, in some embodiments, acontrol part on the joy stick may be made active whenever the mouse iswithin a certain distance or overlap with the part. In this way, thepart that is closest to the mouse is active so that it is moreconvenient for a user to operate to control via a particular part of thejoy stick. In addition, the active control part of the joy stick may bemade visual visible as to its active status. For instance, the activepart may be highlighted whenever the mouse is close by, so that a usercan more readily see which part of the joy stick is currently active.

Each control part on the joy stick may have its own operational range.For example, in some embodiments, the operational range of the stickbase may be the surface of the 3D plane to which the joy stick isattached. The range to perform a sliding operation along the stick bodymay be the length of the stick body, which may or may not have a fixedlength. The length of the stick body may have a normalized relationshipwith the thickness of the 3D volume in a direction that is perpendicularto the surface of the 3D plane to which the joy stick is attached. Inthis way, when a mouse slides up and down along the stick body, eachsliding position along the stick body corresponds to a particular depthin that direction. In some embodiment, the length of the stick body maycorrespond to the portion extended from the stick base along a directioncoming out of the 2D display screen. In some embodiments, the stick bodymay extend from the stick base in two opposite directions, one is in adirection coming out of the screen and another direction going into thescreen. This is illustrated in FIG. 4. In this case, the total length ofthe stick body may have a normalized relation with respect to thethickness of the 3D volume in the direction perpendicular to the surfaceof the plane to which the joy stick is attached.

Similarly, the stick tip may also have its operational range. In someembodiments, the operational range of the stick tip (e.g., rotation withrespect to the stick base) may be within the surface area or a partialsurface area of an imaginary sphere. This is illustrated in FIG. 3. Suchan imaginary sphere is centered at the stick base and has a radius equalto the length of the stick body. In some embodiments, the operationalrange of the stick tip may be confined to the visible surface of thesphere, as shown in FIG. 3. In some embodiments, the operational rangeof the stick tip may be extended to the occluded surface of the sphere,as illustrated in FIG. 4.

When the stick tip is dragged, the stick tip intersects with theimaginary sphere. At each position the stick tip is moved to on thesphere, the orientation of the plane is determined based on the tangentplane at the position of the stick tip. In some embodiments, theorientation of the plane may be parallel to the tangent plane. In someembodiments, the orientation of the plane and the tangent plane may havea certain relationship. In this manner, a user may rotate the plane in aflexible way.

In some embodiments, when the orientation of the plane to which the joystick is attached changes, the orientations of other orthogonal planesmay also change in order to maintain their spatial relationship witheach other. For instance, if there are three planes orthogonal to eachother, when the orientation of one changes, the orientations of otherplanes change accordingly so that the orthogonality among three planesis maintained. In some embodiments, a user may have the choice todesignate the three viewing areas 107 a, 108 a, and 109 a to displaycross sectional views obtained by corresponding cutting planes. In thiscase, when the orientation of a plane changes, not only the orientationsof other planes also change, the corresponding cross sectional viewscorresponding to each of the planes also change. Therefore, the 2Dimages displayed in the 2D viewing areas 107 a, 108 a, and 109 a as wellas the 2D image displayed in display area 102 a may be dynamicallyupdated on the fly while the orientations of the planes undergo changes.In this manner, a user is given a range of flexibility to explore the 3Ddata.

In some embodiments, when the orientation of the plane to which the joystick is attached (e.g., 102) changes, the rendering of the joy stickmay also be dynamically updated. For instance, the joy stick may beattached to the plane, e.g., 102, in a manner so that the stick body andthe surface of the plane forms a certain spatial relationship (e.g., thestick body is parallel to the surface normal of the plane). In thiscase, whenever the orientation of the plane is changed, the 3D post ofthe joy stick may need to be changed accordingly in order to maintainthe spatial relationship.

As illustrated above, the flexible movement of individual parts of thejoy stick allows a user to explore the 3D data in an effective manner.In addition, since the 2D cross sectional view corresponding to acutting plane is also displayed in the same 3D rendering space, a userdoes not need to go back and forth between 3D control and 2D control orshift the focal point in order to understand the 3D data. When multipleplanes are controlled by the same joy stick and cross sectional viewsfor these planes are designated to be displayed in the viewing areas,this also provide a user a powerful tool to visualize and understand the3D data. Furthermore, the ability to extend the operations of the stickbody and stick tip to the occluded part of the 3D data, as describedherein, also enhance a user's capability to effectively explore andmanipulate the 3D data.

As discussed, in some embodiments, a movement of a control part of thejoy stick, e.g., a rotation of the stick tip, a sliding on the stickbody, or a position shifting of the stick base, may be implemented orrealized through the use of 2D mouse movements. This makes commoditycomputers or devices more versatile and provides a cost effective way toenable enhanced capability in 3D data exploration and manipulation. Tofacilitate mouse enabled joy stick 3D control without relying on virtualtrack ball (which is not common and more expensive), a mouse position asoccurred on a 2D display screen needs to be transformed into a 3Dposition in a 3D space where the 3D data and the joy stick isvisualized.

An enabling aspect that makes 2D-based mouse action capable ofcontrolling a 3D structure relates to a transformation from a 2D screencoordinate (where the mouse is) to a 3D coordinate in 3D scene. Such aconversion may be carried out in the following manner. Thetransformation may involve a first conversion from a 2D screencoordinate to a 2D canvas coordinate as well as a second conversion froma 2D canvas coordinate to a 3D coordinate. The first conversion may beneeded due to different coordinate conventions. A 2D screen coordinatesystem may have a different raster scan convention than that of a 2Dcanvas coordinate system. For example, the 2D screen coordinate systemmay have the top-left corner as its origin while the 2D canvascoordinate system may have the lower-left corner as its origin. Inaddition, the 2D screen coordinate system may have a different scopethan the 2D canvas coordinate system. For instance, the 2D canvas mayhave a coverage that is only a part of the 2D screen.

When any of the such situations exists, a 2D screen location may need tobe converted to a 2D canvas coordinate. Depending on the differencebetween the 2D screen coordinate system and the 2D canvas coordinatesystem, a 2D screen coordinate may be horizontally flipped to derive acorresponding 2D canvas coordinate. Alternatively, a 2D screencoordinate may be vertically flipped to derive a corresponding 2D canvascoordinate. In other situations, a 2D screen coordinate may betranslated or transformed in terms of a combination of translation androtation to derive a corresponding 2D canvas coordinate.

A 2D canvas coordinate, represented by (x,y), may then be transformed toa 3D coordinate in a 3D space. In some embodiments, a 3D coordinatecorresponding to a 2D canvas coordinate (x,y) may be identified bytracing a shoot ray until intersecting with a 3D relevant surface. Theshooting ray may be in a direction as defined by a vector that isconstructed using two 3D points, (x,y,0) and (x,y,1), where the “0” and“1” represent the depth dimension and (x,y,0) corresponds to a 3D pointcloser to the screen and (x,y,1) is a 3D point that is farther away fromthe screen.

The 3D structure to be intersected may be determined based on which partof the joy stick is currently activated or being manipulated. Forexample, when the mouse is used to drag and move the stick tip to changethe orientation of the plane to which the joy stick is attached, the 3Dstructure to be intersected is a sphere defined by the joy stick. Whenthe mouse is used to move the stick base, the 3D structure to beintersected is the planar surface of the plane. When the mouse is usedto slide along the stick body, the 3D structure to be intersected is thestick body itself. When the underlying 3D structure is known, the raytracing can be performed along the direction of the shooting ray untilit intersects with the 3D structure at interest. The intersection pointdefines a 3D position in 3D space having a 3D coordinate correspondingto the transformation of the 2D canvas coordinate from which the tracingis initiated.

In some embodiments, such intersection point may be derivedanalytically. For instance, a 2D canvas point can be transformed into a3D point on a sphere when the analytical representation of the sphere isknown. The relationship between a 3D scene and 2D screen is analogous toviewing a 3D world through a camera. Assuming there is a transformationT between a 3D coordinate system to a 2D canvas coordinate system. Toidentify an intersection point between a shooting ray originated from a2D canvas coordinate corresponding to a mouse position on a 2D screenand the sphere defined by the joy stick as described herein, assume thatthe sphere in the 3D space is centered at a known 3D position where thestick base is and has a radius r so that the sphere can be analyticallyrepresented by equation ( P− C)=r². As described herein, the radius ofthe sphere 302 may correspond to the length of the stick body of the joystick. Further assume that the 2D canvas coordinate of the current mouseposition is (x, y). A shooting ray can be constructed along a vector ina direction from an in-canvas point (x,y,0) towards an out-of-canvaspoint (x,y,1).

The two points (x,y,0) and (x,y,1) may then be inverse transformed usingthe transformation T and normalized to generate normalized 3Dcoordinates, denoted by P1 and P2, respectively. A transformeddirectional vector V may then be represented as V=P2−P1. A line extendedfrom P1 along the direction V may intersect with the sphere in two, one,or zero point. When there is no intersection, it means that the mouse isnot in control of the stick tip. When there is only one intersectingpoint, the shooting ray intersects the sphere at a tangent point and theshooting ray is on the tangent plane of the sphere at that intersectingpoint. When there are two intersecting points, the shoot ray shootsthrough the sphere.

Denote an intersection point as P3, which can be analytically solvedusing equation P3=P1+k*V, where k is the distance from P1 to P3. SinceP3 intersects the sphere, it also satisfies the sphere equation.Therefore, we have ( P 1+k V− C)²=r¹². To solve this equation for k,there may be no solution, corresponding to the situation of nointersecting point, one solution, corresponding to the situation ofhaving the shooting ray on the tangent plane of the interesting point,or two solutions, corresponding to the situation where the shooting raycuts through the sphere. When there are two solutions, it is appropriateto select the solution representing an intersecting point closer to P1or closer to the viewer's eye. If there is no solution, the mouse is notin control of the joy stick and therefore no control part of the joystick can be moved by the mouse. Consequently, in this case, the mousehas no control over the 3D data and display thereof. To obtain control,the mouse needs to be moved to a different location so that the mouse isover or sufficiently close to a control part of the joy stick.

Similarly, an intersecting point between a shooting ray and a planarsurface may also be analytically solved. Such a analytical solution maybe applied to identify the 3D coordinate of a mouse location on theplane to which the joy stick is attached. If such identified 3Dcoordinate coincides with the 3D coordinates of the stick base orsufficiently close, the mouse is in control of the stick base and can beused to move the stick base from one location to another on the plane towhich the joy stick is attached.

In the same fashion, an intersecting point between a shooting ray andthe stick body may also be analytically solved depending on theanalytical representation employed for the stick body. For example, insome embodiments, the stick body may be represented as a cylinder in the3D space. Then the intersecting point may be derived by solving anequation for a cylinder by substituting the representation of P3 in theequation for the cylinder. In some embodiment, the stick body may alsobe represented as a 3D line to simplify the solution. In this case,instead of seeking an analytical solution, a point on the shoot ray thathas the minimum distance to the 3D line may be identified and examined.An intersecting point may be identified only if the minimum distance issmaller than some threshold.

While the inventions have been described with reference to the certainillustrated embodiments, the words that have been used herein are wordsof description, rather than words of limitation. Changes may be made,within the purview of the appended claims, without departing from thescope and spirit of the invention in its aspects. Although theinventions have been described herein with reference to particularstructures, acts, and materials, the invention is not to be limited tothe particulars disclosed, but rather can be embodied in a wide varietyof forms, some of which may be quite different from those of thedisclosed embodiments, and extends to all equivalent structures, acts,and, materials, such as are within the scope of the appended claims.

1. A method, comprising: providing a 3D rendering space in a firstregion of a 2D screen; and rendering 3D data in the 3D rendering spacein a first 3D orientation, wherein the 3D data are rendered with one ormore cross sectional views of the 3D data derived based on correspondingone or more 3D planes cutting through the 3D data at a 3D location, afirst plane of the one or more 3D planes has a joy-stick renderedthereon and has a 3D pose determined based on the joy-stick.
 2. Themethod of claim 1, wherein the joy-stick comprises a stick base, a stickbody, and a stick tip.
 3. The method of 2, wherein the stick base of thejoy-stick is rendered with respect to the first plane.
 4. The method ofclaim 2, wherein the stick body is extended from the stick base in a 3Ddirection so that the stick body and the first plane form a certainangle.
 5. The method of claim 4, wherein the stick tip is on an end ofthe stick body opposite of the stick base along the stick body.
 6. Themethod of claim 2, wherein at least one of the stick base, stick body,and stick tip of the joy-stick is active.
 7. The method of claim 6,wherein the active part of the joy-stick is determined based on alocation of a mouse operable in a 2D space.
 8. The method of claim 7,wherein the active part of the joy-stick changes when the 2D screenlocation of the mouse changes.
 9. The method of claim 6, wherein theactive part is highlighted.
 10. The method of claim 6, wherein theactive part can be moved graphically in the 3D rendering space.
 11. Themethod of claim 10, wherein a movement of an active part in the 3 Drendering space is achieved by grabbing and/or dragging the active partusing a mouse operable in the 2D screen.
 12. The method of claim 10,wherein the joy-stick is rendered as a rigid object so that a movementof one part of the joy-stick causes other parts to move accordingly as arigid body.
 13. The method of claim 2, wherein a movement of the stickbase causes a change in a 3D location on the first plane to produce anupdated 3D stick base position based on which the joy stick is rendered.14. The method of claim 13, wherein a movement of the stick base causesa change in rendering the joy-stick in accordance with the updated 3Dstick base position.
 15. The method of claim 2, wherein a movement ofthe stick tip causes a change in an orientation of the first plane toproduce an updated 3D orientation of the first plane.
 16. The method ofclaim 15, wherein a movement of the stick tip causes a change inrendering the joy-stick in accordance with the updated 3D orientation ofthe first plane.
 17. The method of claim 2, wherein a mouse movementalong the stick body causes a change of a position of the first plane toproduce an updated 3D location of the first plane with respect to the 3Ddata.
 18. The method of claim 17, wherein a movement of the stick bodycauses a change in rendering the joy-stick in accordance with theupdated 3D location of the first plane.
 19. The method of claim 15,wherein the movement of the stick tip causes the orientations of the oneor more 3D planes to change.
 20. The method of claim 17, wherein themovement of the stick body causes the positions of the one or more 3Dplanes to change.
 21. The method of claim 13, wherein the updated 3Dstick base position is determined through a transformation from a 2Dscreen location of a mouse position on the 2D screen to a 3D coordinaterepresenting the updated 3D stick base position in a 3D coordinatesystem.
 22. The method of claim 21, wherein the transformationcomprises: converting the 2D screen location to a 2D canvas location;transforming the 2D canvas location to the 3D base position representedas a 3D coordinate in a 3D coordinate system.
 23. The method of claim22, wherein the converting comprises: determining a first 2D coordinatecorresponding to the 2D screen location in a first 2D coordinate system;determining a first transformation between the first 2D coordinatesystem to a second coordinate system associated with a 2D canvas;transforming the first 2D coordinate to a second 2D coordinate in thesecond coordinate system using the first transformation, where thesecond 2D coordinate represents the 2D canvas location.
 24. The methodof claim 23, wherein the transformation comprises: constructing a vectorbetween a first 3D point, represented by a first 3D coordinate, and asecond 3D point, represented by a second 3D coordinate, in the 3Dcoordinate system; tracing a shooting ray in a direction along thevector until intersecting with the first plane; determining a third 3Dpoint as the intersection point with the first plane; and determiningthe 3D coordinate of the third 3D point in the 3D coordinate system forto represent the updated 3D stick base position.
 25. The method of claim24, wherein the first 3D coordinate is constructed based on the second2D coordinate and a first value representing a first depth in 3Dcoordinate system; and the second 3D coordinate is constructed based onthe second 2D coordinate and a second value representing a second depthin 3D coordinate system, where the second value is larger than the firstvalue and represents a deeper depth into the 2D screen.
 26. The methodof claim 15, wherein the updated 3D orientation of the first plane isdetermined in accordance with a tangent plane at an intersecting 3Dpoint on a 3D sphere in a 3D coordinate system, where the sphere isdetermined based on a first 3D coordinate representing the 3D positionof the stick base and a second 3D coordinate representing the 3Dposition of the stick tip before the movement.
 27. The method of claim26, wherein the 3D sphere has a radius measured as the distance betweenthe stick base and the stick tip in the 3D coordinate system, which canbe determined based on the first 3D coordinate and the second 3Dcoordinate.
 28. The method of claim 26, wherein the intersecting 3Dpoint is determined by: transforming a 2D screen location of a mouse toa mouse 3D coordinate, where the mouse is operable in the 2D screen andused to rotate the stick tip in the 3D rendering space; constructing avector between the mouse 3D coordinate and a third 3D coordinate;tracing a shooting ray in a direction along the vector untilintersecting the sphere for the first time at the intersecting 3D point.29. The method of claim 28, wherein the third 3D coordinate has the samecoordinate as the mouse 3D coordinate except a value representing thedepth of the third 3D coordinate, where the third 3D coordinate has avalue representing a deeper depth.
 30. The method of claim 28, whereinthe intersecting 3D point is solved analytically as P3=P1+k*V, where P1represents the mouse 3D coordinate, V represents the vector, krepresents a distance from P1 to P3, where k can be solved based on anequation (P1+k*V−C)=r², where C is the 3D stick base positionrepresented by the first 3D coordinate and r represents the radius ofthe sphere.
 31. The method of claim 30, further comprising selecting,when more than one solutions satisfy the equation, one of the more thanone solution that corresponds to a k representing a distance closer tothe mouse 3D coordinate.
 32. The method of claim 17, wherein the updated3D location of the first plane is determined in accordance with a 3Dintersecting point that intersects the stick body in a 3D coordinatesystem and is originated from a first 3D coordinate representing a 3Dlocation of a mouse operable in the 2D screen and is used to slide alongthe stock body in the 3D rendering space.
 33. The method of claim 32,wherein the 3D intersecting point is determined by: transforming a 2Dscreen location of the mouse to the first 3D coordinate; constructing avector between the first 3D coordinate and a second 3D coordinate;tracing a shooting ray in a direction along the vector untilintersecting with the stick body at the 3D intersecting point.
 34. Themethod of claim 1, wherein the joy-stick has a predetermined operablerange in the 3D rendering space.
 35. The method of claim 1, wherein thejoy-stick is operable on both sides of the first plane.
 36. The methodof claim 1, further comprising displaying, in the 3D rendering space, a2D image of a cross sectional view obtained by cutting through the 3Ddata using the first plane.
 37. The method of claim 1, furthercomprising a second region in the 2D screen.
 38. The method of claim 37,wherein the second region comprises one or more 2D viewing areascorresponding to the one or more 3D planes, respectively.
 39. The methodof claim 38, wherein each of the viewing areas is used to display one ormore 2D cross sectional views obtained by slicing through the 3D data inan orientation determined based on a 3D plane corresponding to the viewarea.
 40. A method for medical data processing, comprising: providing amedical data processing system having a 3D data processing tool, whereinthe 3D data processing tool is capable of: defining a 3D rendering spacein a first region of a 2D screen; and rendering 3D data in the 3Drendering space in a first 3D orientation, wherein the 3D data arerendered with one or more cross sectional views of the 3D data derivedbased on corresponding one or more 3D planes cutting through the 3D dataat a 3D location, a first plane of the one or more 3D planes has ajoy-stick rendered thereon and has a 3D pose determined based on thejoy-stick.
 41. A method for computer aided design, comprising: providinga computer aided design system having a 3D data processing tool, whereinthe 3D data processing tool is capable of: defining a 3D rendering spacein a first region of a 2D screen; and rendering 3D data in the 3Drendering space in a first 3D orientation, wherein the 3D data arerendered with one or more cross sectional views of the 3D data derivedbased on corresponding one or more 3D planes cutting through the 3D dataat a 3D location, a first plane of the one or more 3D planes has ajoy-stick rendered thereon and has a 3D pose determined based on thejoy-stick.